PR Summary

This PR introduces several important enhancements aimed at improving dependency management, testing, and error handling in the project. Key changes include:

1. Dependency Testing Workflows:

   • A new GitHub Actions workflow (.github/workflows/dependency-testing.yaml) has been added to test the compatibility of various dependency sets across multiple Python versions and dependency modes (default vs max). This workflow also includes a pip freeze testing job to validate client environments via the workflow_dispatch event and scheduled runs.
   • The existing Python workflow has been updated to install extras using a consolidated command (poetry install --all-extras) rather than separate ones.

2. Pyproject.toml and Packaging Updates:

   • The package metadata has been updated to use the new PEP 621 style in pyproject.toml. Dependency groups have been refined, and several optional dependencies have been reorganized under descriptive keys (e.g., nlp, stats, credit_risk, explainability).
   • The version check mechanism in the package has been updated: the code now uses importlib.metadata to retrieve the installed version, with a graceful fallback if the package metadata is missing.

3. Enhanced Error Handling for Missing Dependencies:

   • Numerous modules throughout the project (tests, dataset loaders, model validators) now wrap external dependency imports in try/except blocks. When an import fails, a custom MissingDependencyError is raised with a clear message and guidance on how to install the required extra dependencies (e.g., pip install validmind[nlp]).
   • This change improves the developer and user experience by providing actionable feedback for missing libraries in different contexts (stats, NLP, credit risk, etc.).

4. Minor Adjustments in Notebooks and Documentation:

   • Notebook samples have been updated to reflect correct import names (e.g., changing plotly_express to plotly.express) and minor metadata changes (e.g., nullifying execution count in code cells) to improve reproducibility and clarity.
   • The Makefile and other scripts have been slightly adjusted to ensure compatibility with these changes, particularly in API documentation generation.

Overall, these changes strengthen the dependency testing strategy, improve error transparency for missing dependencies, and modernize the project packaging structure.

Test Suggestions

• Run the new dependency-testing GitHub Actions workflow to verify compatibility across all specified Python versions and dependency configurations.
• Trigger the pip-freeze testing job manually via the workflow_dispatch interface to ensure that pip freeze files are correctly handled.
• Test the version check functionality by installing the package in different environments (with and without package metadata available) to confirm the logic works as expected.
• Simulate missing dependency scenarios (e.g., uninstall a dependency like 'scorecardpy' or 'datasets') to verify that the appropriate MissingDependencyError is raised with clear instructions.